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ABSTRACT 



Increased attention on the quality of teaching has meant an 
increase in the use of student surveys as a measure of perceived teaching 
quality. Many student surveys, which typically take the form of teacher and 
course evaluations, are conducted at the end of a school term and are more 
summative in nature. Results are given to the instructor after the class is 
concluded making it impossible to incorporate feedback into the current 
course. This paper shows how the authors used the anonymous survey feature in 
Blackboard at Louisiana State University in Shreveport to survey students at 
midterm in order to allow teachers to collect formative feedback and to make 
crucial changes if indicated. A faculty workshop that provided hands-on 
assistance with importing the survey into Blackboard classes is described, 
and results of a survey of faculty who attended the workshop are summarized. 
Directions for installing the Blackboard survey and a copy of the student 
survey itself are appended. (Contains 15 references.) (Author/MES) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



n IR021388 



Using Blackboard to Survey Students at Midterm 



G\ 

1— H 

VO 

VO 

rl- 



Donna Austin 

Coordinator for Academic Technology 
Associate Professor 
Management & Marketing Department 
LSU in Shreveport 
One University Place 
Shreveport, LA 71 1 1 5 
(318)797-5021 
daustin@pilot.lsus.edu 



John Austin 
Professor 

Economics and Finance Department 
LSU in Shreveport 
One University Place 
Shreveport, LA 71 1 1 5 
(318)797-5022 
iaustin@pilot.lsus.edu 



Abstract 



Increased attention on the quality of teaching has meant an increase in the use of 
student surveys as a measure of perceived teaching quality. Many student surveys, 
which typically take the form of teacher and course evaluations, are conducted at the 
end of a school term and are more summative in nature. Results are given to the 
instructor after the class is concluded making it impossible to incorporate feedback into 
the current course. We will share how we used the anonymous survey feature in 
Blackboard to survey our students at midterm in order to allow our teachers to collect 
formative feedback and to make crucial changes if indicated. 
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Introduction 



The ultimate purpose of any student evaluation of teaching is the improvement of 
teaching, and implicit in that is the improvement of student learning (Ballantyne, 1997). 
While the literature on student evaluation of teaching is constantly growing, it usually 
focuses on the instruments used, with very little about the mode used to administer the 
survey. At Louisiana State University in Shreveport, a mid-term survey for students was 
introduced as an option to help instructors make necessary changes in their classroom 
practices during that same semester. While our faculty feel student surveys conducted 
at the end of the term are extremely important, many voiced this concern: What can I be 
doing "now" to improve my teaching and my student's learning? Another problem with 
end-of-term surveys is that sometimes the most recent impressions are the most vivid 
for students so they might not be reflecting on the whole semester when completing 
such a survey. This is another reason to introduce a mid-term survey — to find out about 
a wider spectrum of the semester rather than only the end of the term. 

Before we came up with this idea of using Blackboard's survey feature to conduct mid- 
term surveys, faculty members had no structured feedback mechanism with which to 
make changes in their teaching style. The results from our end-of-semester surveys 
(Student Instructional Report-ll) are not returned until at least a month into the following 
semester. Additionally, these surveys are only administered in the fall for tenured faculty 
members unless the faculty members request that their students be surveyed in the 
spring as well. 



Review and Background 

The literature that has been published on student evaluations of teaching is vast and 
many works give detailed reviews of what has been covered (Cashin, 1990; Centra, 
1993; Ramsden, 1992). A review of the literature indicates many articles in this genre 
are generally concerned with how feedback is given to faculty (Brinko, 1993) how 
faculty deal with it (Busuttil, 1995) or how useful it is (Marsh & Roche, 1994). Most 
studies consider particular aspects - reliability and validity of questionnaires, biases that 
might occur, etc. It is not the focus of this paper to examine these areas, other than to 
note that the consensus of opinion is that student ratings are generally valid, reliable 
and free of bias (Ballentyne, 1997). Evaluation is generally the last step during the 
semester process. According to Ballentyne (1997), few studies look at what happens 
after the evaluation as far as reporting the results to the students. A prerequisite 
condition for teachers to make improvements to their teaching as a result of student 
feedback is that they consider student opinion worth listening too. Respect, care for 
students, and listening to what they have to say on teaching and other issues are 
considered fundamental aspects of good teaching (Brookfield, 1986; Centra, 1993; 
Greene, 1973; Taylor, 1995; Vella, 1994). 

In fall 2001 we constructed a 28-question survey whose content we will discuss next. 
The survey questions were typed into Blackboard using the Pool Manager so that it 
could be put into an exportable format, which is a zipped file, enabling it to be used by 
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any faculty member who chose to participate. We had two hands-on workshops in 
February 2002 so that faculty could actually import the survey into their Blackboard 
classes. A detailed instruction sheet (See Appendix-1) was provided at the workshop as 
well as an instructor to help with each step. We worked through the various steps of 
importing the zipped file into the various Blackboard classes. The next step was to use 
the Assessment Manager to modify the survey and make it available for students. 

During this step, faculty members could add, change or delete questions from the 
survey. They could also use it just like it was originally designed. 

Eighteen faculty members attended the two workshops in February. This is about 13% 
of our full-time faculty. Only 8 instructors (6%) have actually administered the survey in 
their classes at this point. Four more say they are going to administer it soon, and we 
presume six are not going to administer it. All of the instructors who have given the 
survey have said they will give or have given feedback to their students about the 
pedagogical decisions they took or will be taking as a result of feedback received from 
the survey. Ramsden & Dodds (1989) suggest communicating with students on 
changes that have been made so that students complete future surveys seriously. While 
providing this information to students may reinforce their views of the importance of the 
surveys, this study addresses the mode used to administer the surveys, the ease of 
importing the survey and whether or not the instructor receives helpful information. The 
main issue for the instructors is that this mid-term survey provides evidence of the 
instructor's concern for students by surveying them when changes can still be made in 
their classrooms. 



The Survey Creation 

We created the 28-question survey (See Appendix II) by combining various survey 
questions found on the web (e.g., Murdoch University Student Surveys of Teaching 
questions - http://cleo.murdoch.edu.au/evaluation/survev/teachdraft.html ), along with 
some questions from Northwestern State University’s online course evaluation, and one 
question that pertains to a faculty member’s use of Blackboard. There is one open- 
ended question at the end where students can make comments that relate to teaching 
and learning and the faculty member in general. The student is warned to avoid 
comments that are personal, sexist or racist. The student is also reminded that there 
will be another opportunity at the end of the semester to comment on the class again in 
an anonymous fashion. Students do not take this survey using paper and pencil. 

The students are reminded several times in the instructions and in the Survey 
Announcement Link that the survey is anonymous and that the instructor will not know 
who said what. Results from surveys are reported as percentages. Open-ended 
questions have simple listings of the typed comments. Faculty members are reminded 
not to look at the survey results until an ending survey date has arrived. The instructor 
can see who took the survey, but not what the person said unless there is only one 
person who has answered the survey. On most questions, students answer using the 
following scale with these choices: Strongly Disagree, Disagree, No Opinion, Agree, 
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Strongly Agree and Unable to Judge. Each faculty member can adapt the survey to his 
or her specific discipline, add or delete questions, or leave it in the more generic form. 

Findings 

After mid-term, we surveyed the 18 faculty members who attended the workshop to see 
if they used the survey. We used a simple email survey, and all but two faculty 
members responded. This survey is not meant to be used to show any statistical 
differences; it is simply reporting usage. These are the questions we asked them. 

1 . Did you actually import and use the survey after attending the class or looking at the 
survey in BBDEMO? 

2. Did you give it to your class once or more often? If more often, how often? 

3. Did you change the survey - add or delete questions? 

4. Did you find it easy to follow the directions you were given to IMPORT and TURN 
ON the survey? 

5. If #4 was NO, could you list some of the problems you had. If YES, skip this one. 

6. Did you give extra points to your students to motivate them to answer the survey? 

7. What was your response rate? How many students actually took the SURVEY in 
your various classes? (example: 25/39) 

8. Was the information you obtained from the survey helpful to you in improving 
teaching and learning in your classroom? If you would like to, please explain how you 
used some of the responses and what you did as a result of the responses from your 
students. 

9. Did you communicate to your students that you would use the information from the 
survey to improve teaching and learning? The literature says this is important. 

Eight of the faculty members had actually administered the survey at this time while four 
said they were going to do so soon.[Q1] One had given it more than once.[Q2] Some 
of those who had not yet given the survey or do not plan to give it did not answer some 
of these questions, so various answers will not always add to 16. Six either did not 
change the questions or did not plan to change the questions as given, while five did 
make changes. [Q3] All found the instructions easy to follow.[Q4] Six gave extra credit 
(points) for participation, while four said they would not do so.[Q6] Many did not report 
response rates, but those giving points had in the 70-85% response rates, while the 
others had response rates in the 30-60% range. Nine said they had or would be 
communicating with their students about how the survey results would be used in the 
current class to improve teaching and learning. 

Limitations, Implications and Conclusion 

There are several implications that can be drawn from the present study. First, Black- 
board is fairly easy to use for anonymous surveys, but it is not robust or even very 
usable as far as data analysis is concerned. Results cannot be sent to a statistical 
analysis program such as SPSS or SAS or even a database such as Microsoft Access. 

If you are satisfied with simple percentages for each answer, then Blackboard is 
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sufficient. Second, instructors have to be willing to wait until the time period has expired 
for the students to complete the survey before looking at the results. Otherwise, they 
may be able to figure out what a “lone” person has said. A sophisticated survey 
instrument would not have this problem. Third, students who accidentally get out of 
Blackboard while answering the survey cannot be “reset” to begin again like a quiz. You 
must have some “dummy” names for these students to use in this event. Courses that 
are totally online courses, as opposed to those that use Blackboard as a component, 
should be distinguished as such within the survey and possibly have some unique 
questions. According to Theall (2000), we know very little about the dynamics and 
influence of web-based and distance education vis-a-vis the evaluation of teaching in 
these contexts. The roles, responsibilities, and tasks of teachers and learners in these 
online classes are different from those in face-to-face classes. These differences 
should be explored and addressed in a future Student Survey of Teaching. Finally, 
faculty who give mid-term evaluations and have consultation about them raise their 
scores to the 75 th percentile on evaluations given at the end of the term (Cohen, 1980). 
Faculty who only examine their mid-term survey results raise their scores to the 58 th 
percentile from the 50 th percentile of those giving no mid-term survey. However, this 
paper did not address the use of faculty consultations. Since it is quite obvious from a 
review of the literature that consultation is tremendously important in making 
improvements in student ratings in the end-of-term survey, the workshop to teach 
faculty “how to import” the survey into their Blackboard class should also include 
instruction and help about what to do with the survey results after the fact in order to get 
these kinds of improvements. 
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Appendix - 1 

Directions for getting the survey and installing it in your Blackboard class 

To SAVE to your hard drive the documents you will need: 

1 . Go to the BBDEMO class. 

2. Click on Course Information. Here you will find the zipped file that you will need 
to make the survey for YOUR Blackboard classes. This is file #1 and is called 
Export_StudentSurveyofTeaching.zip. There is also a document that has the 
words prepared for you to use if you want in actually making your survey 
available to your class. It is file #2 under Course Information. 

3. Double-click on the zipped file and SAVE to a folder on your hard drive that you 
will remember. Do the same for the document called “Information to use when 
Preparing Student Survey of Teaching.” 

Now you have two files saved on your hard drive that you will use in the following 

steps. 

To IMPORT this zipped file into your Pool Manager: 

1 . Go to YOUR Blackboard class where you want to give the survey 

2. Click on CONTROL PANEL 

3. Click on POOL MANAGER 

4. Click on IMPORT POOL 

5. Click on the BROWSE button and find the folder where you saved the zipped file. 

6. Double-Click on this file. Then press SUBMIT. 

7. You should see this file on your screen in a list. It’s now been imported. 

8. Click on RETURN TO CONTROL PANEL. 

To make the survey for your class using Assessment Manager: 

1. You should be in Control Panel. Click on ASSESSMENT MANAGER. 

2. Click on CREATE NEW ASSESSMENT. 

3. A box appears that says “Type of Assessment.” There is a dropdown box. Click 
on the dropdown arrow and select SURVEY. Click SUBMIT. 

4. Type Student Survey of Teaching in the first box that says “Enter a Name:” or 
type whatever you want to call your survey. 

5. Type a description in the box that says “Enter a Description.” The next 
paragraph is what I type into this box. [You can edit this paragraph or type your 
own.] The info in this box will appear with the link in the announcement telling the 
student about the survey. [See BBDEMO to see how the announcement link 
looks.] 

Please take this survey for participation points. It is anonymous! I will 
know that you took it, but not what you said. 

6. Click Submit. 

7. The next box says “Enter Instructions.” The next paragraph is what I use here. 
Use this or type your own. 
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This is an anonymous survey. Only I will see the results but I will NOT 
know who said WHAT, only that you responded to the survey. I cannot 
match up your name with your answers. Please respond to the questions 
and press the Submit Answers button at the end when you have 
completed all 28 questions. 

8. Click SUBMIT. 

9. The next screen says, “Select a Question Type.” Click on the dropdown arrow 
and SELECT “From Question Pool or Assessment” at the bottom and then click 
SUBMIT. 

10. Put a check mark to the left of Student Survey of Teaching - 28 questions and 
then click the SUBMIT button at the lower right. You don’t need to preview at this 
time. 

1 1 . The next screen will have “ALL” selected for “Related Categories” and “Select 
Question Type.” Leave those selections as they are and click SUBMIT. 

12. The next screen says “Search Results.” You will need to check (click each box) 
all of the 28 questions. If you do not want to use one of the questions in your 
survey, then do NOT check off that question. Do NOT delete Question #5 since 
that is where I’ve explained what the selections mean, e.g., SA = Strongly Agree. 
You will have to scroll down the page to check all 28 questions. At the bottom of 
this page, click SUBMIT to select all questions for YOUR survey. 

13. At this point, I make the first essay question (it’s really an open-ended question) 
number 28 by clicking on the drop-down arrow with #1 and selecting #28 instead. 

I want the open-ended question to be the last question. I also make the #2 
question about Blackboard usage to be #27. That’s just my preference. 

14. You can add your own questions at this point by click on the “Add New Question” 
button at the upper-left side of the screen. 

15. You are now ready to click on the SAVE AND MAKE AVAILABLE button. 

16. Click on the YES button in the block that says “Make Assessment Available.” 

Leave Generate an Announcement? as it is with YES checked. Leave “Place a 
Link in Assignments” as it is. 

17. DO NOT allow multiple attempts. Each student should answer only once. 

18. Scroll down and click the SUBMIT button. 

19. Your survey has now been created. Click on the Return to Course button. 

To View the Results of the Survey: 

1 . Click on Control Panel 

2. Click on Online Gradebook 

3. Click on Report by Item 

4. Click the dropdown arrow and Select Student Survey of Teaching. Click GO. 

5. Click on VIEW ITEM ANALYSIS 

6. To see the answers to the last open-ended question, go to that question and click 
on VIEW RESPONSES. 

7. Click the BACK button at the bottom left to return to where you were. 

8. Click on RETURN TO COURSE. 
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Appendix - II 

Student Survey of Teaching 

Instructions: This is an anonymous survey. Only your teacher will see the results but 
he or she will NOT know who said WHAT, only that you responded to the survey. He or 
she cannot match up your name with your answers. Please respond to the questions 
and press the Submit Answers button at the end when you have completed all 28 
questions. 

1 . The following were used as resources in this class: (check all that apply) 

a. the textbook 

b. PowerPoint 

c. A computer 

d. A projector hooked to a computer 

e. An overhead projector 

f. The Internet 

g. Other reading materials (articles, etc.) 

h. Other 

2. This class has added to my skills in the following ways: (check all that apply) 

a. Solving problems 

b. Writing papers 

c. Designing lab experiments 

d. Finding trends in data 

e. Critically reviewing articles 

f. Working effectively with others 

g. Giving oral presentations 

h. Learning more about computer programs 

i. Other 

3. In this teacher’s classes, I have gained a good understanding of the concepts 
covered. (The choices are for questions 3 through 26 but will be listed here only 
once.) 

a. Strongly disagree 

b. Disagree 

c. No Opinion 

d. Agree 

e. Strongly Agree 

f. Unable to Judge 

4. This teacher explains the purpose of each class as it relates to the unit. 

5. This teacher is well prepared for class. 

6. Classes by this teacher are well organized. 

7. This teacher communicates effectively with students. 

8. This teacher demonstrates enthusiasm for the subject. 

9. This teacher encourages student participation in class and/or groups. 

10. This teacher provides me with opportunities to apply what I learn. 

1 1 . This teacher encourages me to be responsible for my own learning. 
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1 2. This teacher is sympathetic to student differences. 

13. I found this teacher helpful if I encountered difficulties in this unit. 

14. It is clear what I am expected to learn in each class. 

15. This teacher provides me with useful feedback. 

16. Work graded by this teacher is returned in a reasonable time frame. 

1 7. This teacher marks assigned work fairly. 

18. The course syllabus was clear and unambiguous. 

19. Method(s) of evaluation to be used were clearly explained. 

20. The subject matter covered thus far has been well organized. 

21 . Difficult topics have been clearly explained. 

22. Student questions have been encouraged. 

23. Resources required for the course were available. 

24. Important topics of the course have been included on tests and/or projects. 

25. Feedback about student performance on course assignments has been 
provided. 

26. Instructional quality in this course has met my expectations thus far. 

27. Referring to our course management tool, Blackboard, (check all that apply) 

a. My teacher does NOT use Blackboard 

b. It’s used very little 

c. It’s used a great deal by this teacher 

d. Blackboard helps me to stay on track 

e. Blackboard is a good thing for us to have 

f. I don’t like Blackboard 

28. Comments relating to the teacher can be typed here. There will be an 
opportunity to comment on the class overall at the end of the semester with 
another instrument. This is an anonymous survey. The teacher will see the 
results but won’t know who said what. Please confine your comments to issues 
relating to teaching and learning. Any comments that are personal, sexist or 
racist will be destroyed. Additionally, what are one or two things that this teacher 
does that help you learn? 
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